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Abstract 


In  the  early  2000s  the  U.S.  Navy  decided  to  replace  traditional,  instructor  led 
schoolhouse  training  with  Computer  Based  Training  (CBT).  The  move  was 
expected  to  reduce  training  cost  and  time  without  negatively  affecting  the  quality  of 
sailors  arriving  to  the  Fleet.  If  the  conversion  to  CBT  were  to  have  an  effect 
anywhere  in  the  Navy  maintenance  system,  it  should  be  seen  in  maintenance 
activities  where  sailors  were  performing  maintenance  on  ships.  Anecdotal  evidence 
suggests  that  CBT  failed  to  sufficiently  prepare  new  sailors  for  on  board 
maintenance  and  operations. 

To  determine  the  validity  of  this  claim,  we  examine  data  for  the  AN/SQQ-89(v) 
sonar.  We  analyze  whether  the  US  Navy's  introduction  of  CBT  significantly  affected 
Fleet  maintenance  costs,  actions,  and  training  requirements.  Our  results  suggest 
that  CBT  adversely  impacts  costs,  actions,  and  maintenance  hours  for  the  sonar 
system,  which  seems  to  support  the  anecdotal  evidence. 

Keywords:  system  readiness,  operations,  costs,  computer  based  training 
(CBT),  maintenance 
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Implications  of  Using  Computer-Based 
Training  on  System  Readiness  and 
Operating  &  Support  Costs 


Introduction 

The  majority  of  specialized  skills  training  (known  as  class  ‘A’  and  ‘C’  schools) 
in  the  United  States  Navy  has  traditionally  taken  place  in  a  classroom  setting  with 
live  instruction.  At  the  turn  of  the  century  Navy  leadership  became  concerned  that 
current  training  programs  would  not  adequately  meet  future  demands.  As  a  result, 
the  Chief  of  Naval  Operations  (CNO)  chartered  an  Executive  Review  of  Navy 
Training  (ERNT)  to  review  the  Navy  training  system  and  recommend  solutions  to 
improve  training  effectiveness  and  meet  future  training  demands. 

The  ERNT  group  noted  that  formal  schoolhouse  training  requires  a  large 
investment  in  facilities,  instructors,  and  laboratories  and  that  future  training  demand 
would  exceed  capacity.  The  ERNT  group  recommended  using  new  training 
technologies  to  meet  future  demands  while  reducing  training  costs  and  time 
(Executive  Review  of  Navy  Training  (ERNT),  2001).  Motivated  by  these  findings,  the 
Navy  established  Task  Force  EXCEL  (Excellence  through  Commitment  to  Education 
and  Learning)  to  develop  a  continuum  of  lifelong  learning,  use  a  streamlined  funding 
process  and  a  single  training  authority,  create  a  Human  Performance  Systems 
Model  (HPSM),  and  to  link  training  and  acquisition  (Naval  Personnel  Development 
Command,  2002). 

Part  of  the  Navy’s  new  strategy  included  the  use  of  new  training  technologies 
such  as  distributed  learning,  computer  based  training  (CBT),  collaborative  learning, 
and  computer-mediated  learning.  The  Navy  claimed  that  the  introduction  of  CBT 
would  reduce  both  training  time  and  training  costs  without  reducing  the  quality  of 
training  received  (ERNT,  2001).  Accordingly,  CBT  was  introduced  full-time  into  the 
training  pipeline  in  FY2003. 

A  2009  Naval  Inspector  General  (IG)  Report,  Computer  Based  Training, 
found  that  while  the  introduction  of  CBT  did  reduce  training  time  relative  to  the  A  and 
C  schools,  it  may  not  have  adequately  prepared  sailors  for  their  initial  duty 
assignments.  Anecdotal  evidence  suggested  that  sailors  trained  with  CBT  did  not 
usually  meet  the  required  Knowledge,  Skills,  Abilities,  and  Tools  (KSATs)  to  perform 
their  duties  effectively  upon  reporting  on  board.  Because  of  this,  ships  had  to  take 
the  time  to  train  sailors  up  to  acceptable  standards.  Initial  evidence  also  suggests 


- 1  - 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OE  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


that  the  use  of  CBT  may  have  transferred  these  costs  to  the  operational  fleet  (Naval 
Inspector  General,  2009). 

To  understand  the  operational  cost  impact  of  CBT  we  need  to  know  whether 
CBT  reduced  the  overall  cost  of  operations  and  maintenance,  including  on-the-job 
training,  unscheduled  maintenance  actions,  and  the  length  of  repairs  for  systems 
requiring  intensive  education  and  training. 

In  this  study  we  examine  the  impact  of  CBT  on  a  single  system,  the  AN/SQQ- 
89(v)  sonar\  to  determine  whether  CBT  significantly  altered  fleet  maintenance  costs, 
actions,  and  training  requirements  for  this  system.  If  CBT  has  effectively  trained 
personnel,  then  costs,  labor  hours,  and  corrective  maintenance  actions  should  either 
remain  constant  or  decline.  On  the  other  hand,  if  CBT  is  an  ineffective  replacement 
for  traditional  ‘hands-on’  training,  then,  after  controlling  for  other  factors,  costs,  labor 
hours,  and  corrective  maintenance  actions  should  increase.  We  recognize  that 
focusing  on  one  system  will  limit  the  inferences  about  CBT’s  effect,  however,  we 
maintain  that  these  inferences  will  still  of  be  interest  to  practitioners  and 
policymakers  alike. 

The  next  section  discusses  the  Navy’s  traditional  and  computer  based 
training.  The  third  section  briefly  describes  Navy  maintenance  and  the  fourth  section 
describes  the  AN/SQQ-89(v)  sonar  system  and  the  data  used  in  this  study.  The  fifth 
section  analyzes  whether  CBT  has  affected  maintenance  costs,  actions,  or  time.  The 
last  section  concludes  and  offers  directions  for  future  research. 

Training 

Navy  sailors  receive  training  throughout  their  careers.  Once  a  recruit  has 
completed  basic  training,  he  or  she  will  attend  specialized  skill  training  in  their 
designated  job  specialty  or  “rating.”  In-rate  training  begins  in  a  class  “A”  school,^ 
where  sailors  learn  the  particular  skills  specific  to  their  job.  From  there,  a  sailor  may 
also  attend  additional  in-rate  training  in  a  “C”  school.^  After  completing  school 
training,  the  sailor  is  assigned  to  an  initial  duty  station,  where  training  will  continue 
“on-the-job”  as  he  or  she  gains  real  world  experience  in  their  specialized  skill. 


^  The  AN/SQQ-89(v)  surface  ship  Anti-Submarine  (ASW)  Warfare  combat  system  is  an  integrated 
network  of  sonar  systems  designed  to  search,  detect,  classify,  and  engage  ASW  threats  (Lockheed 
Martin,  2009) 

^  For  further  information  on  qualifications  for  and  assignments  to  class  “A”  schools,  see 
MILPERSMAN  1306-618,  ‘Class  “A”  School  and  Rating  Entry  Requirements.’  Available  at: 
http://www.public.navy.mil/bupers- 

npc/reference/milpersman/1 000/1 300Assignment/Documents/1 306-61 8.pdf 
®  A  submarine  Fire  Control  Technician,  for  example,  attends  a  27  to  33  week  A  school  course  on 
basic  skills,  followed  potentially  by  a  C  school  course  on  advanced  maintenance  topics,  including 
computer  language  skills  and  maintenance  of  specific  weaponry. 
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Additionally,  sailors  can  expect  to  receive  general  military  training  in  topics  ranging 
from  electrical  safety  to  suicide  prevention. 

Traditional  Schoolhouse  Training 

Until  the  early  2000s,  the  Navy  conducted  in  -rate  training  in  a  formal 
schoolhouse  setting  using  subject  matter  experts  (SMEs)  to  instruct  new  sailors. 
These  instructors  typically  came  from  the  Fleet  and  had  practical  experience  with  the 
work  and  responsibilities  of  newly  rated  sailors.  In  addition,  they  were  able  to 
supplement  the  lecture  material  with  tips  and  anecdotes  from  their  own  career 
experiences  (Naval  Inspector  General,  2009). 

In  the  traditional  schoolhouse  environment  sailors  could  reinforce  their 
understanding  of  the  lecture  material  through  hands-on  practice  in  a  laboratory 
setting,  working  on  the  same  equipment  they  would  use  and  maintain  in  the  Fleet. 

In  the  laboratory,  instructors  were  able  to  simulate  equipment  failures  for  technicians 
to  troubleshoot  and  students  were  able  to  ask  questions  as  they  learned.  Because 
the  instructors  were  observing  and  interacting  with  the  students  in  person,  the 
delivery  of  material  (lecture  or  practical  application)  could  be  tailored  to  improve  the 
students’  level  of  comprehension.  For  example,  if  a  class  had  difficulty 
understanding  a  particular  concept,  the  instructor  could  choose  to  spend  more  time 
in  the  lab  to  reinforce  what  is  learned  during  the  classroom  portion. 

There  are  several  benefits  to  Instructor  Led  Training  (ILT).  Since  a  single 
instructor  teaches  a  large  group  of  students,  group  learning  techniques  can  be 
employed  that  would  otherwise  be  unavailable  in  one-on-one  or  CBT  instruction.  The 
formation  of  small  groups  within  a  class  fosters  team-building  and  allows  students  to 
help  and  teach  each  other.  Compared  to  the  costs  of  software  development,  testing, 
and  hardware  purchase,  ILT  is  in  some  ways  more  cost-effective,  depending  on 
class  size  and  length  of  use.  Additionally,  the  controlled  classroom  environment 
offers  fewer  distractions  than  CBT  or  distance  learning.  Finally,  ILT  doesn’t  take  as 
long  to  develop  as  CBT.  It  takes  approximately  34  hours  to  develop  one  hour  of  ILT 
(Chapman,  2007),  while  it  takes  approximately  220  hours  to  develop  a  standard  e- 
learning  course  (Chapman,  2006). 

ILT  also  has  its  disadvantages.  Since  everyone  has  different  learning 
capabilities,  some  students  may  be  more  advanced  and  become  bored  while  waiting 
for  slower  learners  to  catch  up.  Conversely,  slow  learners  may  have  difficulty 
keeping  up.  Depending  on  the  size  and  duration  of  the  course,  ILT  may  be  more 
expensive  than  CBT. 
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Revolution  in  Training 

In  October  2000,  the  Executive  Review  of  Navy  Training  (ERNT)  group  was 
charged  with  providing  insights  on  how  to  improve  and  align  training  organizations, 
leverage  civilian  training  practices,  and  use  new  technologies  to  provide  a  continuum 
of  training  for  sailors.  The  24-member  group  was  comprised  of  military  and  civilian 
personnel,  members  of  academia,  research  institutions,  and  industry.  In  2001, 

ERNT  released  their  report.  Revolution  in  Training:  Executive  Review  of  Navy 
Training  Final  Report. 

During  their  review,  the  ERNT  group  noted  that  the  demands  for  training  had 
increased.  The  ERNT  group  noted  that  the  finite  number  of  seats  available  in  the 
Navy  schoolhouses  would  not  be  able  to  support  the  increased  training  demands. 
Because  of  this,  there  were  gaps  in  the  types  of  training  that  current  and/or  potential 
sailors  needed  and  what  could  be  delivered.  In  many  cases,  this  resulted  in  billets 
which  could  not  be  filled  because  there  were  no  sailors  with  the  required  training  to 
fill  them. 

During  the  1990s  several  other  items  contributed  to  the  lack  of  trained  sailors. 
First,  the  pool  of  experienced  sailors  had  decreased  due  to  drawdowns  and 
retirements.  Second,  it  was  difficult  to  compete  for  trained  personnel  in  a  healthy 
U.S.  economy  and  many  trained  sailors  were  leaving  for  jobs  in  the  civilian  sector. 

The  ERNT  group  suggested  that  technology  and  the  science  of  learning 
offered  several  opportunities  to  improve  the  Navy  training  system  by  reducing 
training  time  through  CBT  and  offering  distributed  learning  opportunities  that  could 
be  executed  at  the  workplace. 

Computer-Based  Training 

Computer-based  training  or  CBT  is  defined  as  “individual  or  group  self-paced 
instruction  using  a  computer  as  the  primary  training  medium,  to  include  web- 
delivered  Navy  E-Learning  (NEL)”  (Naval  Inspector  General,  2009,  p.  ii).  In  Navy  A 
schools,  students  go  through  learning  modules  on  a  personal  computer  at  their  own 
pace.  When  a  student  is  done  processing  the  information  presented  on  the  screen, 
they  click  “next”  to  proceed  to  the  next  piece  of  information.  There  are  usually  small 
knowledge  assessments  throughout  the  module,  followed  by  a  comprehensive 
evaluation  at  the  end  of  the  module  (Naval  Inspector  General,  2009,  p.  7). 

Because  the  learning  is  self-paced,  instructors  were  replaced  with 
“facilitators”  who  are  primarily  concerned  with  maintaining  order  in  the  classroom, 
monitoring  student  progress,  and  providing  technology  assistance.  Facilitators  are 
not  necessarily  SMEs  in  the  subject  matter  being  delivered  in  the  CBT  modules. 
They  do  not  provide  instruction  or  answer  questions  related  to  the  course  content. 
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The  removal  of  instructors  from  the  classroom  may  have  a  detrimental  effect  on 
learning  for  those  students  who  cannot  grasp  the  material  on  their  own. 

The  introduction  of  CBT  in  the  class  A  schools  has  considerably  altered  the 
nature  of  instruction.  Using  a  personal  computer,  sailors  can  progress  through 
learning  modules  at  their  own  pace  or  work  together  in  groups  to  complete  the 
course  material  (Barker,  2010).  In  2010  the  GAO  noted  that  the  fleet  had  concerns 
over  the  level  of  knowledge  that  sailors  reporting  to  ships  from  A  schools  using  CBT 
demonstrated.  The  Navy  IG  noted  that  sailors  arriving  to  the  fleet  under  CBT  did  not 
usually  meet  the  required  Knowledge,  Skills,  Abilities,  and  Tools  standards  (KSATs) 
and  were  unfamiliar  with  the  equipment  they  would  be  working  on  and  the  tools  they 
would  need  to  use.  Because  of  this,  ships  had  to  take  the  time  to  train  sailors  up  to 
acceptable  standards.  In  Fleet  interviews,  some  commands  reported  that  specialty 
qualification  time  was  nearly  double  what  it  had  been  before  the  introduction  of  CBT 
(Naval  Inspector  General,  2009).  GAO  reports  in  2010  and  201 1  made  similar 
observations  and  concluded  that  the  change  to  CBT  had  a  negative  impact  on 
readiness. 

The  Navy  IG  and  GAO  reports  found  that  while  the  Navy’s  use  of  CBT 
resulted  in  cost  and  training  time  savings,  sailors  reporting  to  the  Fleet  were  not  as 
well  prepared  as  ILT-trained  sailors  of  the  past.  The  result  is  that  poorly-trained 
sailors  may  have  contributed  to  declining  material  readiness  in  the  Fleet.  The  next 
section  of  this  study  reviews  Navy  maintenance  practices. 

Maintenance 

Navy  maintenance  occurs  on  three  levels:  organizational  level  (0-level), 
intermediate  maintenance  (IM)  activities,  and  depot  level.  This  section  of  the  study 
will  discuss  all  three  maintenance  levels. 

Shipboard  maintenance  begins  with  the  Planned  Maintenance  System  (PMS). 
PMS  is  governed  by  Naval  Sea  Systems  Command  (NAVSEA)  Instruction  4790. 8B, 
Ship’s  Maintenance  and  Materiai  Management  (3-M)  Manuai.  The  instruction 
outlines  the  requirements  for  PMS  on  shipboard  systems  and  equipment  (Naval  Sea 
Systems  Command,  2003).  The  purpose  of  PMS  is  to  provide  ships  with  the  means 
to  plan,  schedule,  and  perform  preventive  maintenance  onboard  and  to  identify 
potential  equipment  problems  before  the  equipment  fails. 

If  corrective  maintenance  is  required,  the  maintenance  is  reported,  scheduled, 
and  performed  through  organizational  level  (0-level)  shipboard  maintenance.  Ship 
maintenance  actions  are  reported  in  Navy  Visibility  and  Management  of  Operating 
and  Support  Costs  (VAMOSC),  under  Unit  Level  Consumption  and  Manhours — 
Organizational  Corrective  Maintenance. 
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Intermediate  maintenance  (IM)  is  “normally  performed  by  Navy  personnel 
onboard  tenders,  repair  ships,  Shore  Intermediate  Maintenance  Activities  (SIMAs), 
aircraft  carriers,  and  fleet  support  bases.”  (Naval  Sea  Systems  Command,  2003,  pp. 
1-5)  IM  jobs  are  deferred  corrective  maintenance  jobs  that  are  beyond  the  capability 
of  ship’s  force  and  are  sent  off-ship  for  completion.  IM  is  tracked  in  Navy  VAMOSC 
under  Maintenance  -  Intermediate. 

Depot  level  maintenance  “requires  major  overhaul  or  a  complete  rebuilding  of 
parts,  assemblies,  subassemblies,  and  end  items,  including  the  manufacturing  of 
parts,  modifications,  testing,  and  reclamation.”  (Naval  Sea  Systems  Command, 

2003,  pp.  1-5).  Depot  maintenance  is  reported  in  Navy  VAMOSC  under  Maintenance 
and  Modernization  -  Depot,  Other  Depot. 

In  2009  VADM  (Ret.)  Phillip  Balisle  was  directed  to  conduct  a  Fleet  Review 
Panel  (FRP)  of  surface  force  material  readiness.  The  report  noted  that  4,052  billets 
were  removed  from  Navy  ships  from  2001-2009.  While  billets  were  removed  from 
ships,  requirements  such  as  maintenance,  damage  control  watches,  training,  and  in 
port  duties  were  not  reduced  (Balisle,  2010).  The  shortcomings  of  CBT  described  in 
the  previous  section  exacerbated  the  problems  experienced  with  manning 
reductions  since  sailors  were  not  arriving  on  board  with  the  right  KSATs.  The  result 
was  undermanned  ships  with  poorly  trained  sailors  with  not  enough  time  or  know¬ 
how  to  perform  routine  maintenance  actions. 

In  addition  to  reduced  fleet  manning,  shore  facilities  also  received  manning 
cuts.  This  means  that  maintenance  that  was  intended  for  intermediate  maintenance 
activities  was  pushed  back  to  ship  personnel,  which  were  undermanned  and  poorly 
trained.  In  addition  to  the  shrinking  shore  workforce,  the  amount  of  time  the  ships 
are  available  was  shortened  from  15  weeks  to  9  weeks  (Balisle,  2010).  These 
actions  resulted  in  equipment  being  out  of  commission  for  longer  periods  of  time. 

Finally,  the  2010  Balisle  report  noted  that  changes  in  PMS  were  made 
because  ships  couldn’t  meet  maintenance  requirements  due  to  reduced  manning. 
Maintenance  requirements  were  either  eliminated  or  extended  in  periodicity.  The 
intent  was  to  shift  maintenance  requirements  to  shore  facilities,  but  since  manning 
was  reduced  ashore,  many  requirements  went  away  completely.  The  elimination  and 
extension  of  maintenance  requirements  can  lead  to  more  opportunities  for 
equipment  to  become  inoperable,  resulting  in  degraded  Fleet  readiness  (Balisle, 
2010). 

The  Navy  introduced  several  major  changes  to  training,  maintenance  and 
manning  policies  during  the  early  part  of  the  2000-2010  decade.  The  Balisle  report 
found  that  training  was  a  factor  but  certainly  not  the  only  factor  that  led  to  degraded 
Fleet  readiness.  Manning  reductions  would  have  led  to  cost  savings  in  the  military 
personnel  budget,  but  the  impact  of  the  reductions  may  have  resulted  in 
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maintenance  cost  increases  in  future  budgets  due  to  deferred  maintenance  actions, 
thus  confounding  the  effect  of  CBT.  Similarly,  changes  in  maintenance  policies  may 
have  impacted  maintenance  costs  in  future  years. 

At  a  macro  level  the  impact  of  CBT  is  impossible  to  tease  out  (see  Gibson, 
2012,  for  an  examination  of  Navy  training,  operations  and  maintenance  budgets 
between  2000  and  201 2).  For  this  reason,  this  study  examines  one  system  in 
particular,  the  AN/SQQ-89(v)  sonar  system,  collecting  data  at  a  level  of  detail  that 
allows  for  the  control  of  the  various  variables  that  might  impact  maintenance  costs.. 

The  AN/SQQ-89(V)  Sonar  System 

To  examine  the  effect  of  CBT  on  rising  maintenance  costs,  this  study  will 
focus  on  the  O&S  costs  of  a  single  Navy  system,  the  AN/SQQ-89(v)  sonar  system, 
and  look  at  how  the  conversion  to  CBT  affected  maintenance  costs  in  that  system. 
We  selected  this  system  because  it  is  fielded  throughout  the  operational  fleet  before 
and  after  the  implementation  of  CBT.  Analysis  by  Gibson  in  2012  revealed  that 
technicians  did  not  change  significantly  from  FYs  2000-2010,  effectively  eliminating 
manning  as  a  contributor  for  the  AN/SQQ-89  O&S  costs  and  focusing  the  study  on 
training  and  maintenance. 

The  AN/SQQ-89(v)  surface  ship  Anti-Submarine  (ASW)  Warfare  combat 
system  is  an  integrated  network  of  sonar  systems  designed  to  search,  detect, 
classify,  and  engage  ASW  threats.  The  system  is  currently  installed  on  CG-47  class 
cruisers,  DDG-51  class  destroyers,  and  FFG-7  class  frigates.  The  AN/SQQ-89(v) 
employs  a  variety  of  sensors  that  can  transmit  and  receive  acoustic  data  to  detect 
and  classify  threats. (Jane's  Information  Group,  2010). 

The  AN/SQQ-89  system  consists  of  1 5  different  variants.  Variants  differ 
based  on  the  sensors  chosen  and  the  version  of  each  sensor.  In  this  report  only 
variants  2,  3  4,  6,  7  and  9  were  studied.  These  variants  were  chosen  because  they 
were  on  board  ships  prior  to  the  introduction  of  CBT  into  the  sonar  training  pipeline 
(2003)  and  remained  on  board  after  CBT  was  introduced.  This  allows  for  analysis  of 
ship  maintenance  trends  both  prior  to  and  after  the  introduction  of  CBT. 

All  sonar  technicians-surface  (STGs)  attend  STG  A  school.  At  A  school  they 
learn  the  basic  principles  of  oceanography  and  sound.  Following  A  school,  STGs 
who  are  strictly  operators  report  to  a  Sonar  Qperator  course,  while  maintainers 
attend  C  school,  where  they  learn  the  technical  skills  required  to  maintain  the 
equipment  present  on  their  reporting  ship  (Navy  Personnel  Command,  2012).  CBT 
was  introduced  full-time  into  the  training  pipeline  in  FY2003,  after  the 
recommendations  of  the  ERNT  report  (Naval  Inspector  General,  2009).  This  study 
focuses  on  FYs  1999  through  2010  to  capture  data  prior  to  and  after  the  introduction 
of  CBT. 
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Program  Executive  Office  Integrated  warfare  System  5  (PEO  IWS5)  provided 
a  list  of  ships  equipped  with  the  AN/SQQ-89(v)  sonar  system.  The  list  included  ship 
class,  ship  name,  hull  number,  homeport,  and  89  variant  number.  Only  ships  with 
AN/SOO-89(v)  variants  on  board  both  before  and  after  implementation  of  CBT  were 
considered.  The  initial  list  provided  by  PEO  IWS5  included  all  ships  of  the  CG-47, 
DD-963,  DDG-51 ,  and  FFG-7  classes.  To  narrow  the  ship  list  to  match  the  scope  of 
our  study,  ships  were  removed  from  the  data  set  if: 

•  The  ship  was  decommissioned  during  the  FY95-06  timeframe 

•  The  ship  received  a  variant  upgrade 

•  The  ship  was  commissioned  FY2000  or  later 

•  The  ship  was  outfitted  with  a  variant  introduced  after  FY03 

Using  the  above  criteria,  the  ship  list  was  reduced  to  68  ships.  A  list  of  ships 
per  variant  is  given  in  Table  1 . 

Table  1.  List  of  Ships  and  AN/SQQ-89(v)  System  Variants  used  in  this 

study 


(V)2 

SHIP 

HOMEPORT 

(V)3 

SHIP 

HOMEPORT 

(V)6 

SHIP 

HOMEPORT 

CG  55 

LEYTE  GULF 

Norfolk,  VA 

CG  56 

SAN  JACINTO 

Norfolk,  VA 

CG  68 

ANZIO 

Norfolk,  VA 

FFG8 

MCINERNEY 

Mayport,  FL 

CG  57 

LAKE  CHAMPLAIN 

San  Diego,  CA 

CG  69 

VICKSBURG 

Mayport,  FL 

FFG  28 

BOONE 

Mayport,  FL 

CG  58 

PHILIPPINE  SEA 

Mayport,  FL 

CG  70 

LAKE  ERIE 

Pearl  Harbor,  HI 

FFG  29 

STEPHEN  \A/ GROVES 

Mayport,  FL 

CG  71 

CAPE  ST  GEORGE 

San  Diego,  CA 

FFG  32 

JOHN  HALL 

Mayport,  FL 

(V)4 

SHIP 

HOMEPORT 

CG  72 

VELLA  GULF 

Norfolk,  VA 

FFG  33 

JARRET 

San  Diego,  CA 

DDG  51 

ARLEIGH  BURKE 

Norfolk,  VA 

DDG  52 

BARRY 

Norfolk,  VA 

FFG  36 

UNDERVYOOD 

Mayport,  FL 

DDG  53 

JOHN  PAUL  JONES 

San  Diego,  CA 

FFG  38 

CURTS 

San  Diego,  CA 

DDG  54 

CURTIS  WILBUR 

Yokosuka,  Japan 

FFG  39 

DOYLE 

Mayport,  FL 

DDG  55 

STOUT 

Norfolk,  VA 

FFG  40 

HALYBURTON 

Mayport,  FL 

DDG  56 

JOHN  S.  MCCAIN 

Yokosuka,  Japan 

FFG  41 

MCCLUSKY 

San  Diego,  CA 

DDG  57 

MITSCHER 

Norfolk,  VA 

FFG  42 

KLAKRING 

Mayport,  FL 

DDG  58 

LABOON 

Norfolk,  VA 

FFG  43 

THACH 

San  Diego,  CA 

DDG  59 

RUSSELL 

Pearl  Harbor,  HI 

FFG  45 

DE  \A/ERT 

Mayport,  FL 

DDG  60 

PAUL  HAMILTON 

Pearl  Harbor,  HI 

FFG  46 

RENT7 

San  Diego,  CA 

DDG  61 

RAMAGE 

Norfolk,  VA 

FFG  47 

NICHOLAS 

Norfolk,  VA 

DDG  63 

STETHEM 

Yokosuka,  Japan 

FFG  48 

VANDEGRIFT 

San  Diego,  CA 

(V)7 

SHIP 

HOMEPORT 

DDG  64 

CARNEY 

Mayport,  FL 

FFG  49 

ROBERT G  BRADLEY 

Mayport,  FL 

CG  66 

HUE  CITY 

Mayport,  FL 

DDG  65 

BENFOLD 

San  Diego,  CA 

FFG  53 

HAWES 

Norfolk,  VA 

CG  67 

SHILOH 

Yokosuka,  Japan 

DDG  66 

GONZALEZ 

Norfolk,  VA 

FFG  55 

ELROD 

Norfolk,  VA 

DDG  67 

COLE 

Norfolk,  VA 

FFG  56 

SIMPSON 

Mayport,  FL 

(V)9 

SHIP 

HOMEPORT 

DDG  68 

THE  SULLIVANS 

San  Diego,  CA 

FFG  57 

REUBEN  JAMES 

Pearl  Harbor,  HI 

FFG  37 

CROMMELIN 

Pearl  Harbor,  HI 

DDG  69 

MILIUS 

San  Diego,  CA 

FFG  58 

SAMUEL  B  ROBERTS 

Mayport,  FL 

FFG  50 

TAYLOR 

Mayport,  FL 

DDG  70 

HOPPER 

Pearl  Harbor,  HI 

FFG  59 

KAUFFMAN 

Norfolk,  VA 

FFG  51 

GARY 

San  Diego,  CA 

DDG  71 

ROSS 

Norfolk,  VA 

FFG  60 

RODNEY  M.  DAVIS 

Everett,  WA 

FFG  52 

CARR 

Norfolk,  VA 

DDG  72 

MAHAN 

Norfolk,  VA 

FFG  61 

INGRAHAM 

Everett,  WA 

FFG  54 

FORD 

Everett,  WA 

DDG  73 

DECATUR 

San  Diego,  CA 

DDG  74 

MCFAUL 

Norfolk,  VA 

DDG  75 

DONALD  COOK 

Norfolk,  VA 

DDG  76 

HIGGINS 

San  Diego,  CA 

DDG  77 

O'KANE 

Pearl  Harbor,  HI 

DDG  78 

PORTER 

Norfolk,  VA 
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Navy  Visibility  and  Management  of  Operating  and  Support  Costs  (VAMOSC) 
provided  O&S  cost  data,  underway  steaming  days,  and  selected  non-cost  data  for 
ships  equipped  with  the  AN/SQQ-89  sonar  system  covering  FYs  1995  through  2010. 
Cost  figures  were  given  in  then-year  and  constant  FY201 1  dollars. 

In  addition  to  the  AN/SQQ-89  sonar  system  data,  detailed  ship  data  was 
available  for  the  selected  ships.  Non-cost  data  included  number  of  personnel 
trained,  maintenance  manhours,  and  number  of  maintenance  actions.  Table  2 
shows  the  variables  used  in  our  analysis  and  table  3  shows  the  descriptive  statistics 
for  the  data. 


Table  2.  Variables  and  data  sources 


Name 

Definition 

Units  of 
Measurement 

Source 

Organizational  Parts 
Costs 

Cost  of  repair  parts 
used  by  the  ship’s 
personnel  in 
system  maintenance 
and  alterations 

Constant  FY  1 1 
Dollars 

Naval 

Visibility  and 
Management  of 
Operating  and 
Support  Costs 
(VAMOSC) 

Corrective 

Maintenance  Actions 

Number  of  corrective 
maintenance  actions 
performed  onboard 
ship 

Actions 

VAMOSC 

Organizational  Labor 
Hours 

Corrective 
maintenance  hours 
performed  onboard 
ship 

Hours 

VAMOSC 

Authorized  on  Board, 
El  through  E6 

Sailors  authorized 
on  a  ship  for  a  fiscal 
year 

Individuals 

Navy  Personnel 
Command 

Navy  Manning  Plan, 

El  through  E6 

Sailors  planned  for  a 
ship  a  fiscal  year 

Individuals 

Navy  Personnel 
Command 

Currently  on  Board, 

El  through  E6 

Sailors  on  board  a 
ship  at  end  of  a 
fiscal  year 

Individuals 

Navy  Personnel 
Command 

Days  Underway 

Days  spent  at  sea 

Days 

VAMOSC 
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Table  3. 

Descriptive  statistics 

Standard 

Series 

N 

Mean 

Deviation 

Minimum 

Maximum 

Organizational  Parts  Costs 

802 

8439.80 

9434.82 

0 

65839.71 

Corrective  Maintenance  Actions 

793 

75.10 

54.05 

4 

447 

Organizational  Labor  Hours 

801 

1012.60 

1029.57 

7 

12079 

Authorized  on  Board,  E1  through  E6 

808 

12.66 

3.66 

6 

18 

Navy  Manning  Plan,  E1  through  E6 

812 

12.8 

4.09 

5 

20 

Currently  on  Board,  E1  through  E6 

811 

12.92 

4.13 

4 

24 

Days  Underway 

543 

135.80 

48.77 

24 

281 

A  graphical  analysis  by  Gibson  in  2012  indicated  noticeable  changes  after  the 
introduction  of  CBT.  For  example,  Labor  Ashore — Intermediate  Maintenance 
Manhours  showed  significant  change  (see  Figure  1).  The  figure  shows  number  of 
manhours  spent  on  IM  for  selected  ships  from  FYs  1995  through  2010.  Beginning  in 
FY04,  the  IM  manhours  increased  significantly  for  the  selected  DDG-51  and  CG-47 
class  ships. 
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Figure  1 .  Labor  Ashore  -  Intermediate  Maintenance  Manhours 


Analysis  &  Results 

To  test  the  hypothesis  of  whether  the  introduction  of  CBT  significantly 
influenced  system  maintenance  and  operation  we  define  three  dependent  variables 
of  interest:  Organization  Parts  Costs  {OrgParts),  Corrective  Organizational  and 
Intermediate  Maintenance  Actions  {OrgActions),  and  Organization  Labor  Man-Hours 
{Orghours).  If  CBT  does  not  detract  from  sailor  ability,  then  we  expect  CBT  to  have 
no  (or  negative)  impact  on  the  dependent  variables.  If  CBT  does  not  adequately 
prepare  sailors  for  operating  and  maintaining  these  systems  when  compared  to 
traditional  training,  then  we  expect  an  increase  in  parts  costs,  maintenance  actions, 
and  man-hours. 

Computer  Based  Training  (CBT)  is  a  dummy  variable  that  is  0  before  2004 
and  1  afterwards.  Billets  authorized  for  enlisted  grades  E-1  to  E-6  {BAE),  the  Navy 
Manning  Plan  for  enlisted  personnel  in  grades  E-1  to  E-6  {NMPE),  number  of 
enlisted  in  grades  E-1  to  E-6  currently  on  board  (COSE),  and  the  number  of  days 
underway  in  a  given  fiscal  year  {(JW)  are  used  as  control  variables.  A  matrix  Z 
includes  the  radar  variant,  radar’s  installation  year,  type  of  ship,  and  homeport 
location.  The  data  collected  can  be  characterized  as  panel  data  and  the  following 
general  estimation  form  is  used: 


,IL>llWTIAri>!Ql»,m, 
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+  1^57;,+  2BAE,+  ,nmpe,+  ,cobe,+  ,UW,+  Z,+  ,+  ,  +  u,  (1) 


where  iiiand  At  denote  the  unobservable  individual  ship  and  time  effects, 
respectively.  The  term  uit  is  a  random  walk.  The  subscripts  i  and  t  denote  ship  and 
time  period,  respectively. 

To  examine  the  hypothesized  influence  of  CBT  on  AN/SQQ-89(v)  parts  costs, 
maintenance  man  hours,  and  maintenance  actions,  only  those  variants  on  board 
prior  to  and  following  CBT’s  introduction  into  the  A  and  C  schools  are  used.  Of  the 
fifteen  possible  variants,  data  on  five  variants  are  used  for  the  empirical  analysis 
from  FY  1999  through  FY  2010.  The  final  data  set  contains  526  observations  on  68 
ships  from  FY  1999  to  FY  2010. 

Results  from  pooled  Ordinary  Least  Squares  (OLS)  estimators  are  presented 
in  Table  4.  The  analysis  explicitly  assumes  that  CBT  is  exogenous  to  the  dependent 
variables  since  CBT  is  a  policy  decision.  The  results  indicate  that  the  use  of  CBT 
has  adversely  influenced  parts  costs,  actions,  and  labor  hours  associated  with 
operating  and  maintaining  the  AN/SOO-89(v).  This  result  is  consistent  when  we 
control  for  the  type  of  ship,  radar  variant,  homeport,  and  unobservable  ship  and  time 
characteristics.  These  results  suggest  that  the  navy  has  traded  an  explicit  training 
cost  for  an  obscured  cost  in  terms  of  parts,  maintenance  actions,  labor  hours,  and 
readiness. 

Table  4.  Estimates  for  the  impact  of  CBT  (constant  FY11  dollars) 


Pooled 

OLS 

Organizational 
Parts  Costs 

Corrective 

Maint 

Actions 

Organizational 
Maint  Hours 

PartSt-1 

— 

— 

— 

Computer 

Based 

Training 

4971 .45 
(1894.62) 

32.86 

(11.51) 

730.21 

(170.17) 

Billets 

Authorized 

581.46 

(373.73) 

5.18 

(2.53) 

-33.35 

(45.09) 

Navy  Manning 
Plan 

-226.72 

(338.25) 

-3.53 

(2.22) 

4.85 

(39.24) 

Currently  On 
Board 

-332.35^ 

(188.27) 

-0.12 

(0.85) 

17.34 

(19.12) 

Constant 

8159.01 

(2789.05) 

57.75 

(19.80) 

954.13 

(384.28) 

F 

7.58 

16.11 

9.55 

0.20 

0.27 

0.22 

Observations 

790 

779 

787 

M1 

3.24 

4.37 

3.72 

M2 

1.95 

2.26 

1.73^ 

Notes:  **,*,+  denote  significance  at  the  1%,5%,  and  10%  ievei  respectiveiy. 
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Our  analysis  suggests  that  using  CBT  increases  Organizational  Parts  Costs 
by  approximately  $4,971  per  year  at  the  1%  level  of  significance.  For  a  given  system 
on  a  ship,  this  suggests  a  20  to  50%  increase  in  maintenance  costs  over  time.  We 
also  find  that  CBT  increases  Corrective  Maintenance  Actions  by  approximately  32 
per  year  at  the  1%  level  of  significance.  For  a  given  system  on  a  ship,  this  suggests 
a  significant  percentage  increase  in  maintenance  actions.  Finally,  we  estimate  that 
introducing  CBT  inflates  the  number  of  Organization  Labor  Flours  by  730  hours  at 
the  1%  level  of  significance. 

Our  results  support  the  anecdotal  arguments  that  CBT  negatively  impacts 
sailor  performance  on  ships,  affecting  parts  costs,  maintenance  actions,  and 
maintenance  labor  hours.  While  limited  to  one  system,  this  result  suggests  that  the 
navy  has  reduced  the  cost  of  labor  and  equipment  in  schoolhouses  at  the  expense 
of  operational  cost  and  effectiveness  (parts,  maintenance,  and  labor  hours)  on  board 
ships. 

Conclusion 

In  2001,  ERNT  released  its  report.  Revolution  in  Training:  Executive  Review 
of  Navy  Training  Final  Report,  which  led  to  a  major  overhaul  in  the  US  Navy’s 
training  practices,  including  the  use  of  CBT  in  A  and  C  schools.  Anecdotal  evidence 
from  the  Fleet  suggested  that  the  quality  of  training  received  by  sailors  through  CBT 
was  not  as  good  as  the  training  received  in  traditional  schoolhouses.  While 
government  studies  of  the  Navy’s  CBT  training  confirmed  that  the  transition  to  CBT 
resulted  in  shorter  training  times  and  cost  savings,  sailors  reporting  to  the  Fleet  were 
not  as  well  prepared  as  classroom-trained  sailors  of  the  past  and  extensive  OJT, 
supervision,  and  assistance  in  performing  basic  maintenance  tasks  were  required  to 
bring  CBT-trained  sailors  up  to  speed. 

This  study  looked  at  costs  from  a  systems  perspective,  considering  not  only 
the  cost  of  training  but  also  at  the  cost  of  maintenance.  We  asked  the  question;  If 
sailors  trained  with  CBT  had  lower  knowledge  and  skill  levels,  did  this  contribute  to 
increased  operations  and  maintenance  costs? 

Unfortunately  there  were  too  many  confounding  variables  that  could  have 
affected  O&M  costs  during  this  period  of  time  to  draw  any  conclusions  about  the 
effect  of  CBT  on  maintenance  costs  from  the  Navy  level.  Instead,  we  focus  on  a 
single  Navy  system,  the  AN/SQQ-89(v)  sonar  system,  to  examine  the  effects  of  the 
conversion  to  CBT  on  maintenance.  Controlling  for  the  navy's  planning  for  manning 
the  system,  the  number  of  billets  authorized,  and  the  number  of  personnel  on  board, 
we  find  that  CBT  adversely  impacted  costs,  actions,  and  maintenance  hours.  These 
findings  provide,  for  the  first  time  in  the  literature,  empirical  evidence  CBT’s  negative 
impacts,  including  rising  Fleet  maintenance  costs. 
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While  it  seems  reasonable  that  CBT  may  lower  costs  and  maintain  quality  for 
relatively  simple  tasks,  it  may  not  be  as  effective  for  specialized,  knowledge- 
intensive  skills.  Future  research  should  explore  whether  CBT  has  affected  other 
systems  in  a  similar  manner.  This  question  is  of  direct  policy  and  financial  interest  to 
the  navy;  navy  expenditures  may  rise  from  increases  in  costs  and  actions. 
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